[ML] VoyageAI Integration - Clean Version #137519
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- embeddings works, tested - initial rerank code What's missing: - unit and integration tests - rerank request/response mapping and verification
- embeddings works, tested - rerank works, tested (https://www.elastic.co/search-labs/blog/elasticsearch-cohere-rerank) What's missing: - unit and integration tests
- embeddings works, tested - rerank works, tested (https://www.elastic.co/search-labs/blog/elasticsearch-cohere-rerank) What's missing: - unit and integration tests
Moving dimensions to ServiceSettings
elasticsearchmachine
added
v9.3.0
external-contributor
fzowl
force-pushed
the
voyageai-integration-v2
branch
3 times, most recently
from
c480404 to
3c2400a
Compare
- Add text, multimodal, and contextual embeddings support - Add rerank functionality - All code in services/voyageai directory - Includes comprehensive test coverage - Removed 3 files from external/voyageai directory - Removed 3 files from external/http/sender directory - All VoyageAI code now in services/voyageai directory only
fzowl
force-pushed
the
voyageai-integration-v2
branch
from
3c2400a to
c2e11ea
Compare
- Add text, multimodal, and contextual embeddings support - Add rerank functionality - All code in services/voyageai directory - Includes comprehensive test coverage - Removed test-voyageai-e2e.sh script - All VoyageAI code now properly organized in services/voyageai only
fzowl
force-pushed
the
voyageai-integration-v2
branch
from
370aba7 to
72f6dd3
Compare
- Add text, multimodal, and contextual embeddings support - Add rerank functionality - All code in services/voyageai directory - Includes comprehensive test coverage - Removed test-voyageai-e2e.sh script - Deleted ALL voyageai files from external directory (22 files) - All VoyageAI code now properly organized in services/voyageai only
* Fix irregular spaces * Update analysis-keyword-repeat-tokenfilter.md * Update search-suggesters.md * Update search-profile.md
- Add multimodal embeddings support with VoyageAIMultimodalEmbeddings* classes - Add contextual embeddings support with VoyageAIContextualEmbeddings* classes - Refactor text embeddings: move to embeddings/text subdirectory for better organization - Remove old embedding classes from embeddings root directory - Update action creators and visitors to handle new embedding types - Update imports in InferenceNamedWriteablesProvider - Add corresponding request/response entities for multimodal and contextual embeddings - Update unit tests for all new and modified components
- Move text embeddings test files to text/ subdirectory - Fix package declarations in test files - Update VoyageAIEmbeddingsResponseEntityTests to use correct API methods - Change .service() to .getService() and add .getConfigurations() - NOTE: Additional test import fixes needed for full compilation
- Update all VoyageAI test imports to use embeddings/text package - Fix VoyageAIMultimodalEmbeddingsTaskSettings constructor calls (add truncation param) - Fix VoyageAIMultimodalEmbeddingsServiceSettings constructor calls (add embeddingType param) - Update test files: VoyageAIServiceTests, VoyageAIActionCreatorTests, VoyageAIEmbeddingsActionTests - Update test files: VoyageAIEmbeddingsRequestTests, VoyageAIRequestTests - Update test files: VoyageAIMultimodalEmbeddingsTaskSettingsTests, VoyageAIMultimodalEmbeddingsModelTests - Update test files: VoyageAIMultimodalEmbeddingsRequestTests Remaining: 3 compilation errors to fix
Contributor
Author
|
@davidkyle Can you please take a look? |
elasticsearchmachine
added
Team:ML
Collaborator
|
Pinging @elastic/ml-core (Team:ML) |
davidkyle
requested review from
DonalEvans
and removed request for
davidkyle
Contributor
Author
|
@DonalEvans @jonathan-buttner Can you please take a look? |
Contributor
I will try to get to this PR this week, but I have a lot on my plate right now with other large PRs to review. Thank you for your patience. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR contains the cleaned up VoyageAI integration with the following improvements:
Testing: